Systems and methods for determining intraoperative spinal orientation

ABSTRACT

Systems and methods are disclosed whereby a surface detection system is employed to obtain intraoperative surface data characterizing an exposed surface of the spine. In some embodiments, this intraoperative surface data is registered to segmented surface data obtained from volumetric data of the spine in order to assess the intraoperative orientation of the spine and provide feedback associated with the intraoperative orientation of the spine. The feedback may characterize the intraoperative spinal orientation as a change relative to the preoperative orientation.

CROSS-REFERENCE TO RELATED APPLICATION

This application claims priority to U.S. Provisional Application No.62/358,124, titled “SYSTEMS AND METHODS FOR DETERMINING INTRAOPERATIVESPINAL ORIENTATION” and filed on Jul. 4, 2016, the entire contents ofwhich is incorporated herein by reference.

BACKGROUND

This disclosure relates generally to surgical systems and methods, andmore particularly to systems and methods for spinal surgery.

The vertebral column is composed of a series of articulated overlappingsegments. The function of the vertebral column is to support a personwhile standing, balance the individual in the presence of gravity, andenable locomotion and other useful movements. Deformities of the spineinclude conditions such as idiopathic adolescent scoliosis, congenitalscoliosis, post-traumatic deformities, and other adult spinal deformityincluding post-infective kyphosis.

Spinal deformity correction surgery utilizes devices (primarily screwsand rods) to fixate levels of the spine in a corrected or compensatingposition to restore normal posture. Surgical navigation can be used toaid the positioning of screws and other implants within the spine butprovides relatively little feedback on the intraoperative orientation ofthe spine.

Traditionally, intraoperative CT and/or fluoroscopy can be used toassess the orientation of the spine, but these systems are expensive,require the use of large amounts of ionizing radiation, and arecumbersome to use.

SUMMARY

Systems and methods are disclosed whereby a surface detection system isemployed to obtain intraoperative surface data characterizing an exposedsurface of the spine. In some embodiments, this intraoperative surfacedata is registered to segmented surface data obtained from volumetricdata of the spine in order to assess the intraoperative orientation ofthe spine and provide feedback associated with the intraoperativeorientation of the spine. The feedback may characterize theintraoperative spinal orientation as a change relative to thepreoperative orientation. Alternatively, the feedback may consist ofdisplaying the intraoperative spinal orientation by updating thevolumetric data.

Accordingly, in a first aspect, there is provided a method ofdetermining an intraoperative orientation of a spine, the methodcomprising:

obtaining volumetric image data pertaining to a spine;

processing the volumetric image data to generate multi-level surfacedata characterizing a bone surface of the spine;

processing the multi-level surface data to generate segmented surfacedata on a per-level basis for each level of a plurality of spinallevels;

intraoperatively detecting, with a surface detection subsystem,intraoperative surface data characterizing surface regions associatedwith each spinal level of the plurality of spinal levels;

for each spinal level of the plurality of spinal levels:

-   -   employing volumetric fiducial points associated with said each        spinal level and corresponding intraoperative fiducial points        associated with said each spinal level to perform an initial        registration between the segmented surface data associated with        said each spinal level and the intraoperative surface data, and        subsequently performing a surface-to-surface registration        between the segmented surface data associated with said each        spinal level and the intraoperative surface data, thereby        obtaining a registration transform associated with said each        spinal level; and

employing the registration transforms associated with the plurality ofspinal levels to generate measures associated with an intraoperativespinal orientation, and providing feedback based on the measures.

In another aspect, there is provided a system for determining anintraoperative orientation of a spine, the system comprising:

a surface detection subsystem; and

computer hardware operatively coupled to said surface detectionsubsystem, wherein said computer hardware comprises memory coupled withone or more processors to store instructions, which when executed by theone or more processors, causes the one or more processors to performoperations comprising:

processing volumetric image data pertaining to a spine to generatemulti-level surface data characterizing a bone surface of the spine;

processing the multi-level surface data to generate segmented surfacedata on a per-level basis for each level of a plurality of spinallevels;

controlling said surface detection subsystem to intraoperatively detectintraoperative surface data characterizing surface regions associatedwith each spinal level of the plurality of spinal levels;

for each spinal level of the plurality of spinal levels:

-   -   employing volumetric fiducial points associated with said each        spinal level and corresponding intraoperative fiducial points        associated with said each spinal level to perform an initial        registration between the segmented surface data associated with        said each spinal level and the intraoperative surface data, and        subsequently performing a surface-to-surface registration        between the segmented surface data associated with said each        spinal level and the intraoperative surface data, thereby        obtaining a registration transform associated with said each        spinal level; and

employing the registration transforms associated with the plurality ofspinal levels to generate measures associated with an intraoperativespinal orientation, and providing feedback based on the measures.

A further understanding of the functional and advantageous aspects ofthe disclosure can be realized by reference to the following detaileddescription and drawings.

BRIEF DESCRIPTION OF THE DRAWINGS

Embodiments will now be described, by way of example only, withreference to the drawings, in which:

FIG. 1 shows an example system for determining the intraoperativeorientation of the spine and generating feedback associated therewith.

FIG. 2A illustrates an example multi-level surface generated bythresholding volumetric image data of the spine to determine a surfacecorresponding to bone, showing the pre-selected spinal level that isexpected to correspond to a selected intraoperatively exposed spinallevel. The figure also shows three volumetric fiducial points located atthe pre-selected spinal level.

FIG. 2B illustrates an example segmented surface, obtained by segmentingthe multi-level surface of FIG. 2A at the pre-selected spinal level (asidentified by the volumetric fiducial points).

FIG. 2C illustrates an intraoperative surface detected using a surfacedetection system, showing several intraoperatively exposed spinallevels. Three intraoperative fiducial points, corresponding to thevolumetric fiducial points, identify the intraoperatively selectedspinal segment that is believed to correspond to the pre-selected spinallevel in the volumetric frame of reference.

FIG. 3A illustrates the process of shifting the volumetric fiducialpoints via the inter-level transform, in order to generate adjacentvolumetric fiducial points at an adjacent spinal location.

FIG. 3B demonstrates an example method of “snapping” the shiftedvolumetric fiducial points onto the adjacent segmented surface.

FIG. 4A is a flow chart illustrating an example method of generatingfeedback associated with the intraoperative orientation of the spinebased on intraoperative surface detection.

FIG. 4B is a flow chart illustrating an example method of generatingsegmented surface data and volumetric fiducial points for a set ofspinal levels in the volumetric frame of reference, based on volumetricfiducial points identified at a pre-selected spinal level.

FIG. 4C is a flow chart illustrating an example method of generatingintraoperative fiducial points for a set of spinal levels in theintraoperative frame of reference, based on intraoperative fiducialpoints identified at a selected spinal level.

FIG. 5A illustrates the use of inter-level transforms among adjacentlevels in order to generate, based on a set of selected volumetricfiducial points associated with a selected level, additional volumetricfiducial points associated with additional levels.

FIG. 5B illustrates a multi-level surface generated based on volumetricimage data, showing the segmentation of surfaces associated withdifferent levels to obtain per-level segmented surface data.

FIG. 6A shows an example of feedback presented on a user interface,where the example feedback is provided in the form of a center of massand a posterior direction of the preoperative and intraoperative spinalorientations for each spinal level.

FIG. 6B shows an example of feedback presented on a user interface asshown in FIG. 6A, with the addition of numeric values denoting thechange in distance of the center of mass and the angle of the posteriordirection between the preoperative and intraoperative spinalorientations for each spinal level.

FIG. 6C shows an example of feedback presented on a user interface asshown in FIG. 6B, where the change in angle is relative to auser-defined axis.

FIG. 7A shows an example of feedback presented on a user interface,where the intraoperative spinal orientation is displayed by updating thesegmented surface data to match the intraoperative spinal orientation.

FIG. 7B shows an example of feedback presented on a user interface asshown in FIG. 7A, where additional spinal levels are shown byextrapolating the orientation of the spine beyond the intraoperativelyexposed spinal levels.

FIG. 8 shows an example graphical user interface, wherein the positionand orientation of spinal levels from the volumetric data can be definedand adjusted by the user by dragging an orientation vector and moving apoint.

FIG. 9A shows an example graphical user interface, wherein the positionand orientation of spinal levels from the volumetric data can be definedand adjusted by the user by adjusting oriented planes centered at eachspinal level.

FIG. 9B shows an example graphical user interface as shown in FIG. 9A,where the end plates of each spinal level are used as a visual featureto place the oriented planes.

FIG. 10 shows an example of feedback presented on a user interface,wherein the preoperative and intraoperative position and orientation ofspinal levels are displayed, with additional quantification information.

FIG. 11 shows an example of feedback presented on a user interface, inwhich the intraoperative angle between spinal levels is shown.

DETAILED DESCRIPTION

Various embodiments and aspects of the disclosure will be described withreference to details discussed below. The following description anddrawings are illustrative of the disclosure and are not to be construedas limiting the disclosure. Numerous specific details are described toprovide a thorough understanding of various embodiments of the presentdisclosure. However, in certain instances, well-known or conventionaldetails are not described in order to provide a concise discussion ofembodiments of the present disclosure.

As used herein, the terms “comprises” and “comprising” are to beconstrued as being inclusive and open ended, and not exclusive.Specifically, when used in the specification and claims, the terms“comprises” and “comprising” and variations thereof mean the specifiedfeatures, steps or components are included. These terms are not to beinterpreted to exclude the presence of other features, steps orcomponents.

As used herein, the term “exemplary” means “serving as an example,instance, or illustration,” and should not be construed as preferred oradvantageous over other configurations disclosed herein.

As used herein, the terms “about” and “approximately” are meant to covervariations that may exist in the upper and lower limits of the ranges ofvalues, such as variations in properties, parameters, and dimensions.Unless otherwise specified, the terms “about” and “approximately” meanplus or minus 25 percent or less.

It is to be understood that unless otherwise specified, any specifiedrange or group is as a shorthand way of referring to each and everymember of a range or group individually, as well as each and everypossible sub-range or sub -group encompassed therein and similarly withrespect to any sub-ranges or sub-groups therein. Unless otherwisespecified, the present disclosure relates to and explicitly incorporateseach and every specific member and combination of sub-ranges orsub-groups.

As used herein, the term “on the order of”, when used in conjunctionwith a quantity or parameter, refers to a range spanning approximatelyone tenth to ten times the stated quantity or parameter.

As used herein, the term “spinal orientation” refers to the six degreesof freedom in which spinal levels can move relative to other spinallevels. Alternatively, it is also referred to as the “orientation of thespine”. As used herein, the six degrees of freedom of each individualspinal level is referred to by the term “position” for the translationalcomponent, and the term “orientation” is used for the rotationalcomponent.

Various example embodiments of the present disclosure provide systemsand methods for determining information pertaining to the orientation ofthe spine during (or after) performing a spinal procedure. During aspinal procedure, at least two spinal levels are typically exposedintraoperatively. These spinal levels are henceforth referred to asintraoperative spinal levels. The intraoperative spinal orientation,which may change due to a spinal intervention (such as the use of screwsand rods to correct for a spinal deformity or pathology), may bedifficult to visualize, since only a small subset of the spine istypically exposed, and since the surgical field of view is typicallycomplicated by the presence of tissue and blood, thus presentingpotential difficulty to the surgeon in assessing the effect of anintervention on the resulting spinal orientation. As a result,intraoperative X-rays are frequently required, which allows the surgeonto visualize anatomical structures much deeper than the surgicalexposure for spinal level confirmation. This increases the surgicaltime, and exposes the operating room staff and patient to ionizingradiation. It is readily apparent that the consequences of the incorrectexecution of a surgical plan, as a result of an inappropriate surgicalcorrection, can have significant negative consequences for patient andthe surgeon.

Various aspects of the present disclosure address this problem byproviding solutions that employ a surface detection system to obtainintraoperative surface data characterizing the exposed surface of thespine. This intraoperative surface data may be compared with segmentedsurface data obtained from volumetric data of the spine in order toassess the intraoperative orientation of the spine and provide feedbackassociated with the intraoperative orientation of the spine. Asdiscussed below, the feedback may characterize the intraoperative spinalorientation as a change relative to the preoperative orientation. Thefeedback may be relative to a spinal orientation obtained via avolumetric imaging modality at an earlier phase of the procedure. Thefeedback may also be relative to another instance in time during thesurgery of a previous intraoperative spinal orientation. The term“intraoperative”, as used herein, refers to an event that occurs duringa surgical procedure or after the conclusion of a phase of a surgicalprocedure. For example, an intraoperative measurement involving thesurface topography of an exposed potion of the spine may occur any timethat the spine is exposed, such as during an interventional phase of asurgical spinal procedure, and after the interventional phase, but priorto closing the surgical incision.

In one example embodiment, segmented surface data is obtained from thevolumetric image data, such that the segmented surface data correspondsto a pre-selected spinal segment that is expected to be exposedintraoperatively during the surgical procedure. The segmented surfacedata from the pre-selected spinal level, and additional segmentedsurface data from other spinal levels, is registered to theintraoperative surface data, achieving efficient registration, on aper-level basis, and thereby facilitating an assessment of theintraoperative spinal orientation (in absolute terms, or relative to thespinal orientation that existed when the volumetric image data wasobtained). As described in detail below, various example methodsdisclosed herein may employ the determination of a set of inter-levelregistration transforms between adjacent levels in the volumetric frameof reference, in order to assist in the registration between segmentedsurface data of the various levels and the intraoperative surface data,thereby potentially improving the efficiency and accuracy of theinferred intraoperative spinal orientation.

Referring now to FIG. 1, an example system is shown for determining anintraoperative orientation of the spine based on intraoperative surfacedetection. The system includes a surface detection system 10 that isoperably interfaced with control and processing hardware 100. Thesurface detection system 10 may be any suitable system for detecting,measuring, imaging, or otherwise determining the surface topography ofone or more objects (such as, but not limited to, a region of an exposedspine of a patient 50) using optical radiation or sound waves (e.g.ultrasound). Non-limiting examples of suitable optical devices includelaser range finders, photogrammetry systems, and structured lightimaging systems, which project surface topography detection light onto aregion of interest, and detect surface topography light that isscattered or reflected from the region of interest. The detected opticalsignals can be used to generate surface topography datasets consistingof point clouds or meshes. Other examples using sound waves fordetermining surface topography can include ultrasonography.

The example system may also include a tracking system 20, which may beemployed to track the position and orientation of one or more medicalinstruments 40. The medical instrument 40 is shown having fiducialmarkers 45 attached thereto, and passive or active signals emitted fromthe fiducial markers 45 are detected by the tracking system 20 (e.g. astereoscopic tracking system employing two tracking cameras). In analternative example embodiment, the position and orientation of amedical instrument may be tracked via a surface detection subsystem 10,such as a structured light detection system, that is employed to detectthe surface profile of at least a portion of the medical instrument, orstructure attached thereto, and to determine the position andorientation of the medical instrument via comparison of the detectedsurface profile with a known surface profile.

As also shown in FIG. 1, a tracked reference frame 55 (e.g. a clamp withfiducial markers provided thereon or attached thereto) may be attachedto the patient and may be tracked by the tracking system 20. Such atracked reference frame 55 may be employed for image guided surgeries.

FIG. 1 also illustrates an example implementation of control andprocessing hardware 100, which includes one or more processors 110 (forexample, a CPU/microprocessor), bus 105, memory 115, which may includerandom access memory (RAM) and/or read only memory (ROM), a dataacquisition interface 120, a display 125, external storage 130, one morecommunications interfaces 135, a power supply 140, and one or moreinput/output devices and/or interfaces 145 (e.g. a speaker, a user inputdevice, such as a keyboard, a keypad, a mouse, a position trackedstylus, a position tracked probe, a foot switch, and/or a microphone forcapturing speech commands).

It is to be understood that the example system shown in FIG. 1 isillustrative of a non-limiting example embodiment, and is not intendedto be limited to the components shown. Furthermore, one or morecomponents of the control and processing hardware 100 may be provided asan external component that is interfaced to a processing device. Forexample, as shown in the figure, one or both of the surface detectionsystem 10 and the tracking system 20 may be included as a component ofcontrol and processing hardware 100, or may be provided as one or moreexternal devices.

Although only one of each component is illustrated in FIG. 1, any numberof each component can be included in the control and processing hardware100. For example, a computer typically contains a number of differentdata storage media. Furthermore, although bus 105 is depicted as asingle connection between all of the components, it will be appreciatedthat the bus 105 may represent one or more circuits, devices orcommunication channels which link two or more of the components. Forexample, in personal computers, bus 105 often includes or is amotherboard. Control and processing hardware 100 may include many moreor less components than those shown.

Control and processing hardware 100 may be implemented as one or morephysical devices that are coupled to processor 110 through one of morecommunications channels or interfaces. For example, control andprocessing hardware 100 can be implemented using application specificintegrated circuits (ASICs). Alternatively, control and processinghardware 100 can be implemented as a combination of hardware andsoftware, where the software is loaded into the processor from thememory or over a network connection.

Some aspects of the present disclosure can be embodied, at least inpart, in software. That is, the techniques can be carried out in acomputer system or other data processing system in response to itsprocessor, such as a microprocessor, executing sequences of instructionscontained in a memory, such as ROM, volatile RAM, non-volatile memory,cache, magnetic and optical disks, or a remote storage device. Further,the instructions can be downloaded into a computing device over a datanetwork in a form of compiled and linked version. Alternatively, thelogic to perform the processes as discussed above could be implementedin additional computer and/or machine readable media, such as discretehardware components as large-scale integrated circuits (LSI's),application-specific integrated circuits (ASIC's), or firmware such aselectrically erasable programmable read-only memory (EEPROM's) andfield-programmable gate arrays (FPGAs).

A computer readable medium can be used to store software and data whichwhen executed by a data processing system causes the system to performvarious methods. The executable software and data can be stored invarious places including for example ROM, volatile RAM, non-volatilememory and/or cache. Portions of this software and/or data can be storedin any one of these storage devices. In general, a machine readablemedium includes any mechanism that provides (i.e., stores and/ortransmits) information in a form accessible by a machine (e.g., acomputer, network device, personal digital assistant, manufacturingtool, any device with a set of one or more processors, etc.).

Examples of computer-readable media include but are not limited torecordable and non-recordable type media such as volatile andnon-volatile memory devices, read only memory (ROM), random accessmemory (RAM), flash memory devices, floppy and other removable disks,magnetic disk storage media, optical storage media (e.g., compact discs(CDs), digital versatile disks (DVDs), etc.), among others. Theinstructions can be embodied in digital and analog communication linksfor electrical, optical, acoustical or other forms of propagatedsignals, such as carrier waves, infrared signals, digital signals, andthe like. As used herein, the phrases “computer readable material” and“computer readable storage medium” refer to all computer-readable media,except for a transitory propagating signal per se.

Embodiments of the present disclosure can be implemented via processor110 and/or memory 115. For example, the functionalities described belowcan be partially implemented via hardware logic in processor 110 andpartially using the instructions stored in memory 115. Some embodimentsare implemented using processor 110 without additional instructionsstored in memory 115. Some embodiments are implemented using theinstructions stored in memory 115 for execution by one or moremicroprocessors, which may be general purpose processors or specialtypurpose processors. Thus, the disclosure is not limited to a specificconfiguration of hardware and/or software.

The control and processing hardware 100 is programmed with subroutines,applications or modules 150, that include executable instructions, whichwhen executed by the one or more processors 110, causes the system toperform one or more methods described in the present disclosure. Suchinstructions may be stored, for example, in memory 115 and/or otherinternal storage. In particular, in the example embodiment shown,registration module 155 includes executable instructions for registeringsegmented surface data (obtained from the volumetric image data 30) withintraoperative surface data that is obtained using the surface detectionsystem 10, and for determining measures and feedback associated with anintraoperative orientation of the spine (e.g. relative to the spinalorientation in the volumetric image data). The registration module 155may also be employed for computing inter-level registration transformsbetween adjacent levels in the volumetric frame of reference, as persome of the example embodiments described below. The navigation userinterface module 160 includes executable instructions for displaying auser interface for performing, for example, image-guided surgicalprocedures.

Various example embodiments of the present disclosure that pertain theintraoperative determination of spinal orientation employ theregistration of segmented surface data (obtained by processingvolumetric image data of the spine) with intraoperative surface data(intraoperatively obtained using a surface detection system; also knownas a surface topography detection system or surface profile detectionsystem). The volumetric image data may be obtained preoperatively,using, for example, imaging modalities such as, but not limited to,computed tomography (CT) and magnetic resonance imaging (MRI).Alternatively, the volumetric image data may be obtainedintraoperatively, for example, using intraoperative CT or intraoperativeMRI.

As described above, in some example embodiments, the spinal orientation,as determined during or after a spinal procedure involving an exposedportion of the spine, may be determined by performing registrationbetween segmented surface data (obtained from volumetric image data) andintraoperative surface data, and employing the resulting registrationtransforms to generate measures, and/or a visualization, associated withthe intraoperative orientation of the spine, where the measures and/orvisualization may, in some example embodiments, pertain to the change inthe spinal orientation relative to the spinal orientation in thevolumetric image data, or relative to another instance in time duringthe surgical procedure.

Referring now to FIG. 4A, an example method is illustrated fordetermining the intraoperative orientation of a spine (at least aportion of the spine that includes the intraoperatively exposed levels)based on intraoperatively acquired spine surface topography data. Instep 300, volumetric image data of the spine is obtained, as describedabove. Multi-level surface data is then obtained by processing thevolumetric image data, as shown at 305, such that the multi-levelsurface data includes at least a plurality of spinal levels that areexpected to be exposed during the surgical procedure.

An example a multi-level surface 210 is shown in FIG. 2A. In themulti-level surface image 210 of the spine, many volumetric spinallevels can be seen, potentially allowing a clear determination of theidentity (i.e. level number) of a given volumetric spinal level. Thismulti-level surface 210, characterized by associated multi-level surfacedata, resides in the volumetric frame of reference that is associatedwith the volumetric image data. The multi-level surface data may begenerated according to a wide variety of methods. One example involvesthe selection of a bone threshold and generating an isosurface using themarching cubes algorithm from the volumetric image data. Another exampleis to construct an isocontour from each 2D slice of a volumetric imagedata based on a bone threshold, and stitching the slices together into a3D surface.

The multi-level surface data 210 is then processed to generate thesegmented surface data associated with each level of the plurality ofspinal levels, as shown at step 310 of FIG. 4A. An example of segmentedsurface data 250 is shown in FIG. 2B, where the segmented surface data250 corresponds to level 220 of FIG. 2A. The segmentation of themulti-level surface data to obtain the segmented surface data may beperformed according to any suitable method. One or more of thevolumetric fiducial points, such as volumetric fiducial points 230A-C inFIG. 2A, may be employed to initiate surface segmentation of a givenlevel. The volumetric fiducial points associated with a given spinallevel may be provided via manual input (e.g. as input received from auser or operator), or automatically generated, as described in furtherdetail below.

Non-limiting examples of surface segmentation methods includenon-template-based methods and methods which utilize anatomical shapemodels. Non-template-based methods can utilize geometrical properties,such as connectivity, surface normals, and curvatures to determine theboundary of the segmented region, or statistical properties, such asvariance from nearby neighboring points on the surface. Methods based onanatomical shape models can utilize a pre-computed atlas of vertebra asa template to perform the segmentation. Both classes of methods can alsobe used in combination. In all these methods, one or more volumetricfiducial points can serve as a seed point to initialize the segmentationprocess. Alternatively, for segmentation methods which are fullyautomatic and operate on the entire volumetric data (which are usuallybased on anatomical atlases), one or more volumetric fiducials can beused to tag the level(s) of interest.

As shown in step 315 of FIG. 4A, intraoperative surface data is obtainedusing a surface detection system such as, but not limited to, astructured light detection system. FIG. 2C shows an example ofintraoperative surface data detected using a structured light detectionsystem. In contrast to the multi-level surface data 210 shown in FIG.2A, the intraoperative surface data only has partial bone exposed. Theintraoperative surface data may be obtained in a single scan or image,such that a single intraoperative surface topography dataset is obtainedincluding multiple spinal levels in the field of view. Alternatively,the intraoperative surface data may be obtained using two or moresurface topography measurements, such that each measurement pertains toone or more spinal level.

Having generated the per-level segmented surface data corresponding tothe plurality of spinal levels in the volumetric frame of reference, thesegmented surface data for each level may be registered to theintraoperative surface data of the exposed spine, as shown in steps 320and 325. This registration may be performed as an initial registrationbased on correspondence, at each level, between per-level volumetricfiducial points and respective per-level intraoperative fiducial points,as shown at step 320 of FIG. 4A. The per-level intraoperative fiducialpoints associated with a given spinal level may be provided via manualinput (e.g. as input received from a user or operator), or automaticallygenerated, as described in further detail below.

After generating the initial registration for each spinal level, asurface-to-surface registration may then be performed for each level,between the per-level segmented surface data and the intraoperativesurface data, thereby obtaining a set of per-level registrationtransforms, as shown at step 325 of FIG. 4A. The registration transformsrespectively map, for each level, the segmented surface in thevolumetric frame of reference to the intraoperative surface data. Itwill be understood that any suitable surface registration method may beemployed to perform registration between surfaces, when performingmethods according to the example embodiments disclosed herein.Non-limiting examples of suitable registration methods include theiterative closest point algorithm, wherein the distance between pointsfrom difference surfaces are minimized.

The registration transforms may be processed to determine measurespertaining to the relative positions and orientations of the spinallevels, as shown at step 330, and these measures may be employed togenerate intraoperative feedback. Such measures may provide the spatialrelationships among spinal levels within the intraoperative frame ofreference, and also the intraoperative changes in the positions andorientations of the spinal levels relative to the spinal level positionsand orientations in the volumetric image data. These measures may beemployed to generate feedback pertaining to the intraoperativeorientation of the spine. As used herein, “intraoperative orientation”may refer to the positions of the spinal levels, and/or the orientationsof the spinal levels.

For example, the intraoperative spinal level position and orientation ofthe exposed spinal levels may be determined by identifying a set ofvolumetric level positions and orientations in the volumetric frame ofreference, each volumetric level position and orientation identifying aposition and orientation pertaining to a given spinal level in thevolumetric frame of reference, and then employing the volumetric levelposition and orientation and the per-level registration transforms todetermine an intraoperative set of intraoperative level position andorientation of the spinal levels. The set of intraoperative levelpositions and orientations may be employed to generate a visualizationof the intraoperative locations of the spinal levels.

The registration transforms may also be employed to determine measuresof the change in orientation of each level from the volumetric frame ofreference to the intraoperative frame of reference (e.g. a set of anglesprescribing the angular change of the spinal level). If the orientationsand positions of the spinal levels in the volumetric frame of referenceare known (e.g. as defined by a per-level point and normal vector), thenthe registration transforms can be employed to determine theintraoperative per-level positions and orientations.

The volumetric level positions and orientations may be determined byseveral different methods, non-limiting examples of which are providedbelow. It will be understood that many different methods may be employedto determine a suitable reference location of a spinal level.

In one example implementation, a volumetric level position may bedetermined by processing the segmented surface data in order todetermine the center of mass of the fiducial set for the spinal level.In some cases, the segmented surface data may be generated such thateach point in the segmented surface data has a normal vector associatedtherewith. In such a case, normal vectors may be obtained bydetermining, for each volumetric fiducial point, an associated closestpoint in the segmented surface data, and then obtaining a correspondingnormal vector for each closest point. The resulting normal vectors maythen be averaged to provide a mean orientation to define the vectorassociated with the orientation of the level. If the segmented surfacedata does not include an associated normal vector for a given closestpoint, then a vector associated with the closest point can be determinedby employing a set of neighboring points to determine a local tangentialplane, thereby obtaining a normal vector associated with the localtangential plane. This method is particularly useful if each of thefiducial set contains fiducials which are selected in a consistentmanner from level to level. For example, a typical fiducial set patternwould consist of one fiducial selected on the center left lamina, centerright lamina and the center of the spinous process. A second fiducialset pattern might consist of the left inferior facet joint, leftsuperior facet joint, right inferior facet joint, right superior facetjoint, inferior tip of spinous process, superior tip of spinous process.

In a second example implementation, each of the fiducial sets may beused as seeds to initiate a region growing process to segment a regionof each level in a similar manner as the segmentation of multi-levelsurface data into segmented surface data. The points within thesegmented regions may then be used to calculate a mean position for eachlevel to define the point. Similarly, the mean orientation of each levelmay be calculated by averaging the normal associated with each of thepoints contained within the segmented regions. This method mayoutperform the method described above when fiducials are notconsistently selected from level to level.

In a third example implementation, a graphical user interface can beemployed to receive input from a user selecting a suitable per-levelreference location. For example, a user may provide input to select(e.g. drag) a point. The input may also permit adjustment of anorientation vector overlaid onto a 3D rendering of the volumetricsurface data to define this information. An example graphical userinterface to do this is shown in FIG. 8.

In a fourth example implementation, a plane can be shown in a graphicaluser interface, enabling the user to manipulate the plane such that itdescribes the orientation of the spinal level. FIG. 9A shows an exampleof such a user interface. A plane 501 that describes the orientation ofa spinal level is shown. To assist the user in manipulating the planes,indicator arrows suggesting a particular direction of the spinal levelcan be shown. For example, plane 501 consists of arrows, 508 and 509, toindicate to the user that 508 should point to the left of the level, and509 should point to the posterior direction of the level. Such planescan be shown at multiple spinal levels, as shown in 502 to 505 for fouradditional levels. In FIG. 9A, the planes are positioned at the centerof each spine level. Alternatively, it may be advantageous instead toposition the plane at the end plate of each spinal level, which iseasier to identify by the user compared to the center of a spinal level.This is shown in FIG. 9B, where planes 511 to 515 defining theorientation of five spinal levels are displayed.

Positioning of the planes may be assisted by also showing the planes in2D views of the image data. As shown in 506, coronal slices of the imagedata showing planes 501 to 505 can be displayed to the user, enablingfurther fine tuning of the planes by interacting with the linerepresentation of the planes in 2D. Sagittal slices 507 can also bepresented to the user to give a different view to fine tune theorientation of the planes. A similar representation is shown in 516 and517, showing the 2D representation of the planes 511 to 515. One or moremethods (such as any of the preceding example methods) may be used incombination for defining a reference location of a spinal level. Forexample, the aforementioned region growing method may be employed as afirst step, followed by receiving user input to further refine theposition and orientation vector of each spinal level before use indetermining the intraoperative level positions.

Once the intraoperative spine orientation has been obtained, the degreeof residual kyphosis, lordosis or scoliosis can be assessed. In someexample embodiments, one or more visualizations may be generated todisplay the intraoperative positions and/or orientations of the spinallevels, optionally compared to the preoperative volumetric positionsand/or orientations of the corresponding spinal levels. For example,deformation of the spine may be visualized by a 3D plot, where alocation and a vector may be used to depict the position of each spinallevel relative to other levels, as shown in FIG. 6A comparing the spinalpositions and orientations as determined from the volumetric frame ofreference 500 to the spinal positions determined in the intraoperativeframe of reference 510. For example, the position can be used torepresent the center of the spinal level (or another suitable referencelocation), and the vector can represent the ‘posterior’ orientation ofthe spinal level. These point and vector pairs may be extracted asdescribed above from the volumetric frame of reference 500, which canthen be transformed by the corresponding per-level registrationtransform to generate the corresponding intraoperative positions and/ororientations 510.

In addition to the position and orientation, additional measures, suchas, but not limited to, the difference in angle of the vector anddisplacement of the point can be displayed, as shown in FIG. 6B. Inanother example embodiment, evaluation of the change in spinalorientation may be assessed via user-guided techniques. For example, itmay be advantageous for the user to define anatomical axes from whichupdated deformity measurements can be made. An example is shown in FIG.6C, where an axis 520 is defined by the user. The generated report ofthe change in angles are made relative to this user-defined axis.

In an alternative embodiment, instead of visualizing a 3D plot, theintraoperative orientation of the spine may be visualized using thevolumetric surface image generated from the volumetric image data, asshown in FIG. 7A, where the positions and orientations of the variouslevels are shown at 405C, 410C, 415C, 420C and 425C. Here, eachsegmented surface data associated with a corresponding spinal level istransformed by the corresponding per-level registration transform. Thisoperation positions and orients each segmented surface data to representthe intraoperative orientation of the spine. This may be more intuitive,as it gives the user better context of each spinal level's position andorientation relative to adjacent spinal levels.

In some deformity surgeries, it may be advantageous to measure anddisplay the angle between adjacent spinal levels intraoperatively toconfirm that the correction of a deformity, such as scoliosis of thespine, has been achieved. Furthermore, it may be advantageous tovisually observe the correction that has been attained during aprocedure and compare that with a preoperative view of the spine. FIG.10 shows an example of such a display, where the intraoperativeorientation of five spinal levels 531 to 535 are shown, and is overlaidon top of the preoperative bone surface 500 of the spine. In this view,the lowest spinal level (towards the feet) 535 serves as an anchor pointwith the corresponding level in the preoperative bone surface of thespine. Alternatively, the intraoperative and preoperative orientation ofthe spine can be shown side by side, without overlay. Planes 501 to 505,as previously defined based on the preoperative spine orientation, areshown in this example, where the plane orientations have been updated tomatch the orientation of the corresponding intraoperative spinal levels.In some example implementations, angles can be calculated between theseplanes. For example, the angle between spinal levels 531 and 532, asdefined by their corresponding vector of the planes projected in thecoronal plane is shown in 521. Similar angles are displayed as 522, 523,and 524. Angles in other planes, such as the axial and sagittal planes,can be similar shown. In another example implementation, a graphicaluser interface 540 can be provided that enables the user to determinethe angle between pairs of spinal levels in the coronal, sagittal, andaxial plane by selecting from a menu, which can be used to measure Cobbangles. An example of such a graphical user interface is shown in FIG.11.

Another example embodiment of assessing the intraoperative orientationof the spine may include the insertion of two or more user-definedmeasurement planes and/or vectors, associated with two or more levels.This can take the form of an adjustable overlay displayed on top of thevolumetric data visualization. This enables measurement of relativeangulation between two or more levels in user-defined planes. Additionalmetrics that may also be extracted include, but are not limited tosacral slope, pelvic incidence, pelvic tilt, sagittal vertical axis andcoronal shift.

Some of the preceding example implementations employ the computedregistration transforms to generate feedback pertaining to changes inthe positions and orientations of the spinal levels from the time atwhich the volumetric image data was acquired to the time at which theintraoperative surface image data was obtained. As noted above, in someimplementations, the volumetric image data may be obtainedpreoperatively. In other example implementations, the volumetric imagedata may be obtained intraoperatively, using an intraoperativevolumetric imaging modality, such that the feedback showing the changesin positions and orientations of the spinal levels as they relate tointraoperative changes. In another example embodiment involvingintraoperative changes, two or more intraoperative surface measurementsmay be obtained, at different times during a surgical procedure, and theaforementioned methods (using registration transforms relative tosegmented surface data obtained based on volumetric image data) may beobtained to determine, for each associated surface measurement, theintraoperative positions and orientations of the exposed spinal levels.The different intraoperative positions and orientations of the spinallevels at these time points may be employed to generate feedbackindicative of intraoperative changes between time points associated withthe intraoperative surface measurements.

It may be advantageous in some cases to show additional spinal levelspresent in the volumetric data that are not intraoperatively exposed.This is shown in FIG. 7B, where the additional set of spinal levels 450and 460 are obtained from the multi-level surface data, and positionedand oriented above and below, respectively, the set of intraoperativelyexposed spinal levels 470. To determine the position and orientation ofthe set of spinal levels 450 and 460 relative to the intraoperativelyexposed region, additional volumetric fiducials are obtained in theadjacent spinal levels above (480) and below (490) the intraoperativelyexposed region. Following the same method described earlier to processthe multi-level surface data to generate segmented surface data, twoadditional inter-level transforms can be determined for these spinallevels adjacent to the intraoperatively exposed region. The set ofspinal levels whose positions and orientations are to be extrapolatedcan be obtained by subtracting the segmented surface data 405C, 410C,415C, 420C, and 425C from the multi-level surface data. The inter-leveltransforms can then be applied to the 450 and 460 accordingly to showtheir extrapolated position and orientations.

In the example embodiment described above and illustrated in the flowchart shown in FIG. 4A, per-level volumetric fiducial points andcorresponding per-level intraoperative fiducial points are employed toregister the segmented surface data, on a per-level basis, to theintraoperative surface data. In one example embodiment, the volumetricfiducial points are obtained based on input from a user or operator. Forexample, a user may employ a user interface to select, on a displayshowing the multi-level surface data, at least three volumetric fiducialpoints for each level. As noted above, surface segmentation of themulti-level surface data to obtain segmented surface data for a givenlevel may be performed using at least one volumetric fiducial point fromthe given level to initialize a region growing surface segmentationmethod.

The per-level intraoperative fiducial points may also be obtained basedon input from a user or operator. In one example implementation, a usermay employ a tracked probe (e.g. a probe having fiducial markersattached thereto that are tracked with a tracking system) to select, viacontact with different locations on the spine, intraoperative fiducialpoints for each level, where the intraoperative fiducial pointscorrespond to the volumetric fiducial points on a per-level basis. Insuch a case, a tracked reference frame attached to the subject (e.g.reference frame 55 shown in FIG. 1) may be employed to compensate forthe motion of the spine during point selection.

In one example embodiment, volumetric fiducial points are obtained for apre-selected level, based on input from a user or operator, and theremaining volumetric fiducial points (and the segmented surface data)are automatically generated for the other spinal levels (e.g. the levelsknown or expected to be intraoperatively exposed). An example of such amethod is illustrated in FIG. 4B.

As shown at step 340 of FIG. 4B, input is received from a useridentifying, in the multi-level surface data, at least three volumetricfiducial points associated with a pre-selected level that is expected tobe exposed during the surgical procedure. For example, as shown in FIG.2A, the multi-level surface 210 is employed for the selection of a setof at least three volumetric fiducial points, shown at 230A-C, at thepre-selected spinal level 220. The volumetric fiducial points 230A-C,which may be selected by an operator on a user interface displaying themulti-level surface data 210, identify the pre-selected spinal level 220that is expected to be exposed during a surgical procedure.

Having identified the volumetric fiducial points 230A-C, the multi-levelsurface data 210 may be processed to generate the segmented surface dataassociated with the pre-selected level 220, as shown at step 345 in FIG.4B. An example of the segmented surface data 250 is shown in FIG. 2B,which also shows the volumetric fiducial points 230. The segmentedsurface data 250 includes surface data corresponding to the pre-selectedlevel 220. Segmentation of the multi-level surface data to obtain thesegmented surface data may be performed according to any suitablemethod. One or more of the volumetric fiducial points may be employed toinitiate surface segmentation.

Having performed surface segmentation of the pre-selected spinal level,the pre-selected spinal level, and its associated segmented surfacedata, is employed for the generation of segmented surface dataassociated with an adjacent spinal level, as shown in steps 350 to 365of FIG. 4B. An example of an adjacent level is shown in FIG. 2A at 220B.Unlike the pre-selected spinal level 220, the adjacent spinal level 220Bdoes not have associated volumetric fiducial points to support surfacesegmentation from the multi-level surface data, or to supportregistration with the intraoperative surface data.

In order to facilitate surface segmentation of an adjacent spinal level,an adjacent volumetric region, such as a bounding box (the region neednot be a rectangular prism) is identified in which to performsegmentation, as shown at step 355. The determination of the adjacentvolumetric region may be made based on a determination of directionalinformation associated with the orientation of the spine, where thedirectional information enables the determination of a direction inwhich to locate the adjacent spinal level. The directional informationcan be a direction which defines the entire spine. Alternatively, thedirectional information can be described by a spline or a piece-wiselinear function to follow the shape of the spine.

This directional information may be obtained according to a variety ofmethods, non-limiting examples of which are provided below. In oneexample implementation, the directional information may be obtained frominformation associated with the volumetric image data, such asuperior-inferior direction provided from the DICOM header. In anotherexample implementation, an axis associated with the orientation of thespine may be determined from principal component analysis. In anotherexample implementation, image processing methods may be applied to thevolumetric image data to extract an estimated shape of the spine.

In one example implementation, a set of local spine axes may bedetermined, thereby providing directional information on a per-levelbasis. A preferential axis is initially determined for segmenting thevolumetric image data. The preferential axis may be determined, forexample, from information associated with the volumetric image data,such a superior-inferior direction provided from a DICOM header, or fromprinciple component analysis. The preferential axis may then be employedto segment the volumetric image data into a series of volumetric slabsthat are arranged along the preferential axis, each of which areanalyzed to locate the spine. The choice of slab thickness depends onthe resolution required for computing the directional information of thespine. On the other hand, if the slab thickness is too thin, theaccuracy of the finding the spine within the slab, and hence derivingthe directional information, may be degraded, due to reduction of signal(e.g. structured belong to the spine) to noise (e.g. the background). Aslab thickness of approximately half of the length of a spinal level istypically suitable.

Various methods can be employed to analyze the slabs in order to derivethe directional information of the spine. One example method can betemplate-based, wherein the slabs are compared to a pre-computed atlasof different vertebra. Alternatively, a user-defined threshold can beused to define a contour and/or isosurface of the bone, from which thevertebra region within the slab can be identified. The vertebra regioncan be identified by performing an iterative search for structures thatresemble the vertebra according to a pre-computed atlas. Alternatively,an atlas-free method can be employed, which utilizes one or morevolumetric fiducial points as a starting point via an iterative search.

For the atlas-free method, an initial volumetric slab segment containingone or more of the volumetric fiducial points is identified. An initialbounding box (or other suitable confining volumetric region) is thendetermined, where the initial bounding box contains, and is centered on,or approximately centered on, one or more of the fiducial points. Thesize of the initial bounding box may be determined, for example, basedon the spatial extent of the segmented surface data associated with thepre-selected spinal level, or based on an estimated spatial extent of anaverage spinal level. This initial volumetric slab segment is processed,within the initial bounding box, to determine an initial center of massof bony structures within the initial volumetric slab segment. Thisprocess may be repeated one or more times, where each time, the boundingbox is re-centered on the most recently identified center of masslocation. The center of mass location may be iteratively refined in thismanner until a pre-selected convergence criterion has been met, such asthe change in the center of mass location between subsequent iterationsis below a threshold.

Once the center of mass corresponding to the spine has been determinedin the initial volumetric slab, an adjacent bounding box may then bedetermined, within an adjacent slab. Since the bounds of a vertebra isapproximately the same within the same patient, the adjacent boundingbox can be of the same size as the bounding box from the initialvolumetric slab, wherein the center of the adjacent bounding box can beinitialized with the center of mass from the initial volumetric slab.This adjacent volumetric slab segment is processed similarly, within theadjacent bounding box, to determine an adjacent center of mass locationwithin the adjacent volumetric slab segment. As noted above, thisprocess may be repeated one or more times, where each time, the boundingbox is re-centered on the most recently identified center of masslocation, iteratively refining the center of mass location until apre-selected convergence criterion has been met.

The above method of finding an adjacent center of mass location in anadjacent volumetric slab segment may then be repeated one or more timesin order to determine center of mass locations within a plurality of thevolumetric slab segments, thereby allowing the determination of a localaxis, based on two or more center of mass locations. In one exampleimplementation, the local axis associated with two neighboringvolumetric slab segments may be employed to locate the bounding boxwithin an adjacent volumetric slab region when performing theaforementioned method.

In situations where the initial preferential axis is significantlydifferent than the directional information of the spine (e.g. due todisease), the computed directional information can be used to againsegment the volumetric image data into a series of volumetric slabs, andthe above iterative center finding method repeated to refine thedirectional information of the spine.

After obtaining the directional information (e.g. global or local), thisinformation may be employed to determine an adjacent volumetric regionwithin which to perform segmentation of the multi-level surface data inorder to obtain the adjacent segmented surface data corresponding to theadjacent spinal level, as per step 355 of FIG. 4B. For example, anadjacent bounding box for segmenting the adjacent spinal level may becentered at a location, relative to one or more of the volumetricfiducial points, which lies along an axis obtained based on thedirectional information, such that the bounding box is expected tocontain the adjacent spinal level. The spatial separation between thecenter of the adjacent bounding box and the one or more volumetricfiducial points may be determined, for example, based on the spatialextent of the segmented surface data associated with the pre-selectedspinal level, or based on reference anatomical data (e.g. atlas data)characterizing an estimated spatial separation between the pre-selectedspinal level and the adjacent spinal level.

The multi-level surface data may then be processed within the adjacentbounding box to generate the segmented surface data associated with theadjacent spinal level, as shown at step 360. As noted above, thesegmentation of the multi-level surface data to obtain the adjacentsegmented surface data may be performed according to any suitablemethod.

An inter-level transform is then determined between the pre-selectedspinal level and the adjacent spinal level, as shown at step 365. Theinter-level transform between the pre-selected spinal level and theadjacent spinal level may be determined by performing registrationbetween the segmented surface data (associated with the pre-selectedspinal level) and the adjacent segmented surface data (associated withthe adjacent spinal level). The inter-level transform between thesegmented surface data of the pre-selected spinal level and the adjacentsegmented surface data is defined by following the pre-computeddirectional information, translating by a distance that is based on thespatial extent of the segmented surface data, or using referenceanatomical data (e.g. atlas data) characterizing an estimated spatialseparation between the initial spinal level and the adjacent spinallevel. Fine-tuning of the registration is then performed by any suitableregistration algorithm. It will be understood that any suitable surfaceregistration method may be employed to perform registration betweensurfaces, when performing methods according to the example embodimentsdisclosed herein. Non-limiting examples of suitable registration methodsinclude the iterative closest point algorithm, wherein the distancebetween points from difference surfaces are minimized.

Having obtained the inter-level transform between segmented surface dataof the pre-selected spinal level and the adjacent segmented surfacedata, the position and orientation of the adjacent spinal level,relative to that of the pre-selected spinal level, is known. Thisprocess of determining the segmented surface data for an adjacent spinallevel, and an inter-level transform from the initial spinal level to theadjacent spinal level, may then be repeated for additional adjacentspinal levels, as shown at step 370. As per step 350, when steps 355-365are performed for the first time, the pre-selected spinal level isemployed as an initial level for determining the segmented surface dataand the inter-level transform to the adjacent spinal level. However, asper step 370, each time steps 355-365 are repeated, the previousadjacent level is employed as the initial level, such that the newlydetermined segmented surface data and the newly determined inter-leveltransform pertains to the next adjacent spinal level. This process isrepeated if other spinal levels, of the plurality of spinal levels thatare intraoperative exposed, reside on the opposing side of thepre-selected spinal level.

After having performed steps 340 to 372, segmented surface data isobtained for each spinal level, and inter-level transforms are obtainedbetween each set of adjacent spinal levels, based on the volumetricfiducial points provided for the pre-selected spinal level. As shown atstep 375, the inter-level transforms may be applied to volumetricfiducial points in order to generate, on a per-level basis, volumetricfiducial points associated with the additional spinal levels.

As a first step, the inter-level transform between the pre-selectedspinal level and the adjacent spinal level may be employed to determinelocations, in the adjacent segmented surface data, of adjacentvolumetric fiducial points. According to this example implementation,and as illustrated in FIG. 3A, the inter-level transform may be appliedto the locations of the volumetric fiducial points 230A-C associatedwith the pre-selected fiducial points in the volumetric frame ofreference, such that the volumetric fiducial points 230A-C aretransformed to the region associated with the adjacent spinal level(FIG. 3A shows volumetric fiducial points 230B and 230C, as volumetricfiducial point 230A is hidden in the view shown).

Since the segmented surface data that is associated with thepre-selected spinal level is different than the adjacent segmentedsurface data associated with the adjacent level, the transformedvolumetric fiducial points 240A-C may not lie within the adjacentsurface data. This effect is illustrated in FIG. 3B, where, for example,transformed points 240B and 240C initially lie above the adjacentsegmented surface 260. In order to bring the transformed points 240A-Cinto the adjacent segmented surface data, the transformed points 240A-Cmay be shifted so that they lie within the adjacent segmented surface,as shown at points 240B′ and 240C′ in FIG. 3B.

For example, this may be achieved by computing a location within theadjacent segmented surface data that is nearest to the transformedpoint, and shifting (“snapping”) the transformed point to this nearestlocation, thereby obtaining the adjacent volumetric fiducial point thatlies within the adjacent segmented surface data. Alternatively, thepoint shifting procedure may be performed by computing the local surfacenormal vector that is directed at the transformed fiducial point, andshifting the transformed fiducial point along the directioncorresponding to this vector. Optionally, in combination with thesemethods of shifting the fiducials, multiple candidate nearest locationson the adjacent segmented surface may be evaluated, wherein the choiceis made on a similarity measure of each candidate to the fiducial on thesegmented data. This similarity measure can be based on surface normalsand curvatures in addition to proximity.

This process of generating adjacent volumetric fiducial points may berepeated to generate the volumetric fiducial points for the nextadjacent spinal level, where the next inter-level transform is appliedto the most recently determined adjacent volumetric fiducial points(e.g. after performing the aforementioned “snapping” process). Thismethod may be repeated to generate the volumetric fiducial points forall of the relevant spinal levels, thereby generating a set of per-levelvolumetric fiducial points. This process is illustrated in FIGS. 5A and5B, where user-identified volumetric fiducial point 415A associated witha pre-selected spinal level 400 is employed to generate per-levelvolumetric fiducial points 405A, 410A, 420A and 425A (shown in FIG. 5A)and per-level segmented surfaces 405B, 410B, 415B, 420B and 425B (shownin FIG. 5B).

As noted above, in one example embodiment, the intraoperative fiducialpoints may be provided manually via input from a user or operator.However, in another example embodiment, the intraoperative fiducialpoints may be obtained for a selected level, based on input from a useror operator, and where the intraoperative fiducial points for theselected level correspond to the volumetric fiducial points defined at acorresponding level in the volumetric reference frame. Theintraoperative fiducial points are then automatically generated for theother spinal levels in the intraoperative reference frame. An example ofsuch a method is illustrated in FIG. 4C.

As shown at step 380 of FIG. 4C, input is received from a useridentifying, volumetric fiducial points associated with a selected levelin the intraoperative frame of reference. In one example implementation,a user may employ a tracked probe (e.g. a probe having fiducial markersattached thereto that are tracked with a tracking system) to select, viacontact with the spine at a selected level, the intraoperative fiducialpoints for the selected level, where the intraoperative fiducial pointscorrespond to the volumetric fiducial points at a corresponding level inthe volumetric frame of reference.

As per step 385, the inter-level transforms, defined among pairs ofadjacent spinal levels in the volumetric reference frame (as explainedabove) may then be employed to generate the intraoperative fiducialpoints for the other spinal levels in the intraoperative frame ofreference. If the volumetric fiducial points for the spinal levels weregenerated automatically, then these inter-level transforms will havealready been computed. If the volumetric fiducial points were definedmanually, then the inter-level transforms in the volumetric frame ofreference may be determined by generating segmented surface data foreach spinal level, using at least one of the volumetric fiducial pointsfor each level to initiate segmentation, and then performing surfaceregistration among adjacent levels, as per the method described above.

As a first step when generating adjacent intraoperative fiducial points,the inter-level transform between the spinal level in the volumetricframe of reference that corresponds to the selected spinal level in theintraoperative frame of reference, and the adjacent spinal level, may beemployed to determine locations in the intraoperative reference frame,of adjacent intraoperative fiducial points. This method operates underthe assumption that even through the spine orientation will likely havechanged in the intraoperative frame of reference relative to the spineorientation in the volumetric frame of reference, the inter-level changebetween adjacent levels will be sufficiently small such that theinter-level transform from the volumetric frame of reference is a validapproximation of the spatial relationship between adjacent levels in theintraoperative frame of reference.

According to this example implementation, the inter-level transform(obtained from the volumetric frame of reference) may be applied to thelocations of the intraoperative fiducial points associated with theregion associated with the pre-selected spinal level in theintraoperative frame of reference, such that the intraoperative fiducialpoints are transformed to the region associated with the adjacent spinallevel, in a manner similar to the illustration in FIG. 3A. Registrationmay then be performed between the adjacent segmented surface data andthe intraoperative surface data, where the adjacent volumetric fiducialpoints and adjacent intraoperative fiducial points are used to performan initial registration, followed by a surface-to-surface registration,as shown at steps 320 and 325 in FIG. 4A, to obtain the per-levelregistration transform.

It is noted that the aforementioned method of generating adjacentintraoperative fiducial points is an approximation, and extending thesefiducial points beyond the adjacent spinal level can lead toaccumulation of errors. Accordingly, in one example implementation, theintraoperative fiducial points may be refined by using the per-levelregistration transform previously computed between the adjacentsegmented surface data and the intraoperative surface data. In thisexample method, the intraoperative fiducials associated with the regionassociated with the pre-selected spinal level in the intraoperativeframe of reference are first transformed into the volumetric frame ofreference, using the per-level registration transform corresponding tothe pre-selected spinal level. The inter-level transform is then used tofurther transform the position of these intraoperative fiducial pointsinto the adjacent spinal level, in the volumetric frame of reference. Asa further refinement, the transformed fiducial points are shifted sothat they lie within the adjacent segmented surface data as previouslydescribed, analogous to the illustration in FIG. 3B. Finally, thefiducials points are transformed back into the intraoperative frame ofreference using the per-level registration transform corresponding tothe adjacent spinal level.

This method may be repeated to generate the intraoperative fiducialpoints for all of the relevant spinal levels, thereby generating a setof per-level intraoperative fiducial points, where errors introduced bythe use of the inter-level transforms are iteratively corrected both byusing the inter-level registration transforms and snapping the pointsinto the intraoperative surface, as described above.

The specific embodiments described above have been shown by way ofexample, and it should be understood that these embodiments may besusceptible to various modifications and alternative forms. It should befurther understood that the claims are not intended to be limited to theparticular forms disclosed, but rather to cover all modifications,equivalents, and alternatives falling within the spirit and scope ofthis disclosure.

1. A method of determining an intraoperative orientation of a spine, themethod comprising: obtaining volumetric image data pertaining to aspine; processing the volumetric image data to generate multi-levelsurface data characterizing a bone surface of the spine; processing themulti-level surface data to generate segmented surface data on aper-level basis for each level of a plurality of spinal levels;intraoperatively detecting, with a surface detection subsystem,intraoperative surface data characterizing surface regions associatedwith each spinal level of the plurality of spinal levels; for eachspinal level of the plurality of spinal levels: employing volumetricfiducial points associated with said each spinal level and correspondingintraoperative fiducial points associated with said each spinal level toperform an initial registration between the segmented surface dataassociated with said each spinal level and the intraoperative surfacedata, and subsequently performing a surface-to-surface registrationbetween the segmented surface data associated with said each spinallevel and the intraoperative surface data, thereby obtaining aregistration transform associated with said each spinal level; andemploying the registration transforms associated with the plurality ofspinal levels to generate measures associated with an intraoperativespinal orientation, and providing feedback based on the measures.
 2. Themethod according to claim 1 wherein the feedback includes avisualization of intraoperative positions and orientations of the spinallevels.
 3. The method according to claim 2 wherein the visualizationassociates, with each spinal level of the plurality of spinal levels, alocation of the level and a vector indicative of the orientation of thelevel, wherein the location and the vector are determined based on theregistration transforms.
 4. The method according to claim 2 wherein thevisualization comprises a three-dimensional image of the spine generatedbased on the registration transforms.
 5. The method according to claim 1wherein the feedback includes one or more parameters quantifying theintraoperative orientation and/or position of the spinal levels.
 6. Themethod according to claim 5 wherein the one or more of the parameterscomprise angles characterizing the intraoperative orientations of thespinal levels.
 7. The method according to claim 6 wherein one or more ofthe parameters comprise angles of the spinal levels.
 8. The methodaccording to claim 6 wherein one or more of the parameters comprisedifferences in angles of adjacent spinal levels.
 9. The method accordingto claim 5 wherein the one or more parameters comprise distancescharacterizing relative intraoperative positions of the spinal levels.10. The method according to claim 5 wherein one or more of theparameters are selected from the group consisting of sacral slope,pelvic incidence, pelvic tilt, sagittal vertical axis and coronal shift.11. The method according to claim 1 wherein the feedback includes avisualization showing changes in the intraoperative orientations andpositions of the spinal levels relative to the orientations andpositions of the spinal levels associated with the volumetric imagedata.
 12. The method according to claim 1 wherein the feedback includesone or more parameters quantifying changes in the intraoperativeorientations and/or positions of the spinal levels relative torespective orientations and/or positions of the spinal levels associatedwith the volumetric image data.
 13. The method according to claim 12wherein the one or more parameters include angles characterizing changesin intraoperative orientations of the spinal levels relative to theorientations of the spinal levels associated with the volumetric imagedata.
 14. The method according to claim 12 wherein the one or moreparameters include distances characterizing changes in intraoperativepositions of the spinal levels relative to positions of the spinallevels associated with the volumetric image data.
 15. The methodaccording to claim 1 wherein the volumetric fiducial points are defined,for each spinal level, according to input from an operator.
 16. Themethod according to claim 1 wherein the intraoperative fiducial pointsare defined, for each spinal level, according to input received from anoperator.
 17. The method according to claim 16 further comprising:employing a tracking system to track the position and/or orientation ofa fiducial marker intraoperatively attached to the spine while obtainingthe input identifying the intraoperative fiducial points associated witheach spinal level to compensate for the motion of the spine.
 18. Themethod according to claim 1 wherein the segmented surface data and thevolumetric fiducial points associated with each spinal level of theplurality of spinal levels are obtained by: (i) obtaining inputidentifying at least three volumetric fiducial points at a pre-selectedspinal level within a volumetric frame of reference associated with thevolumetric image data; (ii) employing at least one of the volumetricfiducial points associated with the pre-selected spinal level to performsegmentation on the multi-level surface data, thereby obtainingsegmented surface data associated with the pre-selected spinal level;(iii) employing the pre-selected spinal level as an initial spinal levelwhen performing steps (iv) to (vi) for a first time; (iv) determining anadjacent volumetric region, within the volumetric frame of reference,that is associated with an adjacent spinal level that is adjacent to theinitial spinal level, (v) performing segmentation on the multi-levelsurface data within the adjacent volumetric region, thereby obtainingadjacent segmented surface data associated with the adjacent spinallevel; (vi) registering the segmented surface data associated with theinitial spinal level to the adjacent segmented surface data, therebyobtaining an inter-level transform between the initial spinal level andthe adjacent spinal level; (vii) repeating steps (iv) to (vi) one ormore times, each time using the previous adjacent level as the initiallevel, to generate segmented surface data and the inter-level transformsassociated with additional spinal levels of the plurality of spinallevels on a first side of said pre-selected spinal level, such that eachinter-level transform is between adjacent spinal levels; (viii)repeating steps (iii) to (vii) if additional spinal levels of saidplurality of spinal levels reside on the other side of said pre-selectedspinal level; and (ix) employing the inter-level transforms and thevolumetric fiducial points associated with the pre-selected spinal levelto obtain volumetric fiducial points associated with the other spinallevels of the plurality of spinal levels.
 19. The method according toclaim 18 wherein employing the inter-level transforms and the volumetricfiducial points associated with the pre-selected spinal level to obtainvolumetric fiducial points associated with the other spinal levels ofthe plurality of spinal levels comprises: (x) applying the inter-leveltransform between the pre-selected spinal level and an adjacent spinallevel to the volumetric fiducial points associated with the pre-selectedspinal level, thereby obtaining estimated volumetric fiducial locationsassociated with the adjacent spinal level; (xi) employing the estimatedvolumetric fiducial locations to determine volumetric fiducial pointsresiding within the segmented surface defined by the segmented surfacedata corresponding to the adjacent spinal level; and (xii) repeatingsteps (x) and (xi) to determine the volumetric fiducial pointsassociated with the additional spinal levels of the plurality of spinallevels.
 20. The method according to claim 18 wherein the intraoperativefiducial points are defined, for each spinal level, according to inputreceived from an operator.
 21. The method according to claim 18 whereinthe intraoperative fiducial points are generated by: obtaining inputidentifying at least three intraoperative fiducial points at a selectedspinal level within an intraoperative frame of reference, wherein theselected spinal level in the intraoperative frame of reference isexpected to correspond to the pre-selected spinal level in thevolumetric frame of reference, and wherein the intraoperative fiducialpoints at the pre-selected spinal level correspond to the volumetricfiducial points at the pre-selected spinal level; and employing theinter-level transforms and the intraoperative fiducial points associatedwith the selected spinal level to obtain intraoperative fiducial pointsassociated with the other spinal levels of the plurality of spinallevels.
 22. The method according to claim 21 wherein employing theinter-level transforms and the intraoperative fiducial points associatedwith the selected spinal level to obtain intraoperative fiducial pointsassociated with the other spinal levels of the plurality of spinallevels comprises: (x) employing the registration transform between thepre-selected spinal level and the selected spinal level to transform theintraoperative fiducial points associated with the selected spinal levelinto the volumetric frame of reference, thereby obtaining transformedintraoperative fiducial points; (xi) applying the inter-level transformbetween the pre-selected spinal level and the adjacent spinal level tothe transformed intraoperative fiducial points, thereby obtainingestimated adjacent fiducial locations associated with the adjacentspinal level; (xi) employing the estimated adjacent fiducial locationsto determine transformed adjacent fiducial points residing within thesegmented surface data associated with the adjacent spinal level; (xii)employing the registration transform associated with the adjacent spinallevel to transform the transformed adjacent fiducial points into theintraoperative frame of reference, thereby obtaining intraoperativefiducial points associated with the adjacent spinal level; and (xiii)repeating steps (x) and (xii) to determine the intraoperative fiducialpoints associated with the additional spinal levels of the plurality ofspinal levels.
 23. The method according to claim 18 wherein thesegmented surface data and the inter-level transforms are determined forspinal levels residing on both sides of the pre-selected spinal level.24. The method according to claim 18 wherein the adjacent volumetricregion is determined, at least in part, based on directional informationassociated with the orientation of the spine in the volumetric frame ofreference.
 25. The method according to claim 24 wherein when performingstep (iv), the adjacent volumetric region is determined by: determininga bounding box associated with the segmented surface data of the initialspinal level; employing the directional information to translate thebounding box to a spatial region expected to contain the adjacent spinallevel.
 26. The method according to claim 24 wherein the directionalinformation is determined employing one of: a DICOM header, principalcomponent analysis, and spinal cord location extraction via imageprocessing.
 27. The method according to claim 24 wherein the directionalinformation is determined by: a) determining a preferential axis forprocessing the volumetric image data; b) segmenting the volumetric imagedata into a series of volumetric slab segments arranged along thepreferential axis; c) identifying a volumetric slab segment containingat least one of the volumetric fiducial points; d) determining aninitial volumetric test region bounding the at least one of thevolumetric fiducial points within the volumetric slab segment; e)determining a center of mass location within the initial volumetric testregion; f) determining, based on the center of mass location, anadjacent volumetric test region within an adjacent volumetric slabsegment, and determining an adjacent center of mass location within theadjacent volumetric test region; g) repeating step f) one or more timesto obtain a set of center of mass locations corresponding to differentvolumetric slab segments; and h) processing the set of center of masslocations to determine the directional information characterizing theorientation of the spine.
 28. The method according to claim 27 whereinwhen repeating step f) to identify an additional adjacent center of masslocation within an additional adjacent volumetric slab segment, anadditional adjacent volumetric test region within the additionaladjacent volumetric slab segment is determined by translating apreviously determined adjacent volumetric test region along a localaxis, wherein the local axis is determined based on two center of masslocations corresponding to volumetric slab segments neighbouring theadditional adjacent volumetric slab segment.
 29. The method according toclaim 27 wherein, when performing steps e) or f), the center of masslocation that is determined is an estimated center of mass location, themethod further comprising: generating a refined center of mass bydetermining a second volumetric test region bounding the estimatedcenter of mass location; and determining the refined center of masslocation within the second volumetric test region.
 30. The methodaccording to claim 29 further comprising repeating the determination ofthe refined center of mass one or more times until a pre-selectedconvergence criterion has been satisfied.
 31. The method according toclaim 1 wherein the segmented surface data associated with one or moreof the spinal levels is obtained by performing region growing on themulti-level surface data.
 32. The method according to claim 1 whereinthe surface detection subsystem is a structured light subsystem.
 33. Themethod according to claim 1 wherein the intraoperative surface data isacquired two or more times during a surgical procedure, and wherein aset of registration transforms are respectively calculated, andrespective feedback is intraoperatively generated, each time that theintraoperative surface data is acquired.
 34. A system for determining anintraoperative orientation of a spine, the system comprising: a surfacedetection subsystem; and computer hardware operatively coupled to saidsurface detection subsystem, wherein said computer hardware comprisesmemory coupled with one or more processors to store instructions, whichwhen executed by the one or more processors, causes the one or moreprocessors to perform operations comprising: processing volumetric imagedata pertaining to a spine to generate multi-level surface datacharacterizing a bone surface of the spine; processing the multi-levelsurface data to generate segmented surface data on a per-level basis foreach level of a plurality of spinal levels; controlling said surfacedetection subsystem to intraoperatively detect intraoperative surfacedata characterizing surface regions associated with each spinal level ofthe plurality of spinal levels; for each spinal level of the pluralityof spinal levels: employing volumetric fiducial points associated withsaid each spinal level and corresponding intraoperative fiducial pointsassociated with said each spinal level to perform an initialregistration between the segmented surface data associated with saideach spinal level and the intraoperative surface data, and subsequentlyperforming a surface-to-surface registration between the segmentedsurface data associated with said each spinal level and theintraoperative surface data, thereby obtaining a registration transformassociated with said each spinal level; and employing the registrationtransforms associated with the plurality of spinal levels to generatemeasures associated with an intraoperative spinal orientation, andproviding feedback based on the measures.